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ABSTRACT 

Data collected from one high school was evaluated to 
exemplify how the psychometric method was applied to the Ford 
Training and Placement Program (FTPP) « Evaluation of the FTFP relied 
on both affective, paper *and-pencil measures and classroom 
observations* It was the concern of this investigation to discover 
differences between Ford and non*Ford subjects* An analysis of 
ccnvariance design, with pretest scores covaried out, revealed few 
significant differences in posttest scores* A difference in three 
personality measures seemed to support the validity of the teacher 
selection process* The lack of other differences between Ford and 
nor- Ford individuals may not reflect shortcomings on the program so 
much as the inappropriateness of the measures used« A tighter 
connection between program and evaluation would have made an ad<H]uate 
summative evaluation possible* (Author/ JA) 
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TOE CLASSICAL PS\-CHa\!ETRIC METHOD Or EVALUATiaN OF FTPP 
By M. Kaller and D. Soltz 



This paper will describe a^ki attempt to evaluate the data 
collected from one high school as a means of exemplifying how the 
psychooietric method was applied to the Ford Program. It should be 
noted that only the final year of operation is considered here. Hence, 
this evaluation of FTPP differs frcm those made in pre\'ious years, 
since earlier evaluations were seen as opportunities to experiment 
and re-evaluate successes and failures in reaching a number of goali. 
Those ••formative" evaluations were intended to •'improve^' rather than 
••n)x>ve/^ The final yearns evaluations viere what the former director 
of research termed •'summative/' ^ The differences in the t\.v typas of 
evaluations arc not so much in what is measured as what use the mea- 
sures are put to. It is hoped that these quantitative data afford at 
least a partial summary of the FTPP. 

The conceptual basis » choice of measures uscd» and original 
plan for evaluation all derive frm Wayne Doyle's work and writings. 
In particular, his •Transactional Evaluation in Program Development 
contains a thorough description of his rationale for c}ioosing tlic 
particular measures he did to evaluate the FTPP. The present paper 
will report some partial results from analysis of the quantitative, 
classical, papcr-and-pcncil instruments and observational datct. Rat 
first, \<o should like to indicate that the breadth of poals articu- 
lated for the program and tlie possible effects of reaching those 
goals on the teachers, students, and others involved far outran thr 



ability of classical psychonetric methods altme to adequately measure 
them. For exanple, six major goals and a number of related » minor 
ones are presented here to give some indication not only of the scope 
of the program^ but of the formidable task of evaluation as u-ell. 
One ijivnediate limit to evaluation is thus the number of variables 
that might be dealt with. Some judicious parsing of possibilities 
for dependent variables had to be accomplished. 

The first goal» developing competencies in the teacher » in- 
volved such possibilities for measurement as self-knowledge and know* 
ledge of the learning processes; the development of skills; an in- 
creased comprehension of Black culture; knowledge of the learner as a 
social and academic entity; knowledge about the classroom. Other 
competencies included understanding the school as a social system; 
loiowledge of the cornunity and the environment that surrounds the 
school; loiowledge of the process of change, and of his subject mat- 
ter; knowledge of how to be an effective group member » and how to 
identify and define problems; and the development of problem analysis 
and decision -making skills. 

A second goal of the program was to facilitate the induc- 
tion of newly trained professionals into the school community. They 
are to ujidorstanJ their rclc in the cadre as well as other roles in 
the school and the coiiminity; knoiv human and material resources 
available; accept others in the school and community and interact 
with them. 

The third goal was to proinotc close relations between i^cJiool 
and coimunity: knowledge of the c:nimunity and its resource:;; aware- 
ness of the comnranities^ expcctatidns aj\d problems; md the proiiotion 
of mutual participation of the cormnvuiity in the school. 
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The fourth goal of the program was the developncnt of prc- 
grams appropriate to specific classes of cadre members. This entails 
'View and appropriate curricula/' 

The fifth goal of the program involved the identification 
of school and coRTiamity problems and to act on them within the pro- 
fessional competence. 

Finally, there v-as the goal of developiiij]^ an "aura of 
shared responsibility" among the cadre, other staff, and the conTOi- 
nity for the educational program of the school. Long-term emplo)Tnent, 
increased conmunii tion, encourajjement of collegial relations, fa- 
culty decisions on matters important to it, the proniotion of accounta- 
bility, snd the mutual support of school and coimmity are involved. 

It is clear that this iripressive list of items is diffi- 
cult to evaluate as a piece. Indeed, a number of different measures 
are necessary to make sense of tlie possibilities for evaluation in- 
herent in the list. We v;ould le interested in learning of any pro- 
gram that managed to achieve so many goals, but such evidence is 
difficult to come by. Even v:ithxit hard data, ho-^N'ever, it is dear 
that many of these goals have been mot. Curricula have been deve- 
loped specific to the FTPP, the cor.-nuiiities have been involved, a3}d 
many other goals reached. But the difficulties in measuring, say 
community participation by cadre members are large. There is also 
a risk that wliatever is measured v;ill not be accepted, by one's peers 
as slgnificiunt data. Ke could have counted, for exa^nple, liow^naiiy 
non-school meetings in the comfrainity the Ford teachers attended, 
but how much this \vould tell us about the quality of educational 
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practice within the school is unclear. Nevertheless, as close re- 
lations between the schools and coin.rjnitief \<cic clearly an ex- 
pected outcone of the original model, such measures vxxxld have 
provided data for evaluating that aspect of the program's appli- 
cation of the model. A good deal of data of this sort may be gar 
nered from the case studies and other non-quantitative materials 
of the program, but the question o£- their effect on the children 
and their teachers may have to be addressed more directly to be of 
value or even answerable. 

Some difficult questions are raised for the evaluater 
intent on obtaining quantitative data! Should he discard the tra- 
ditional achievement data on the grounds that it is irrelevant 
or will show no effect over the short run? This would mean sacri- 
ficing one of the most widely accepted, easily measured set of 
variables. Furthermore, measures in the "affective domain" (as 
Bloom would teim those actually used) tend to have less desirable 
measurement properties than "cognitive" data. Reliabilities are 
often small due to large error variances, and validity is often a 
nightmarish consideration that involves thorny questions of inter- 
pretation, subjectivity of items, difficult comparisons of sub- 
ject groups and so forth. Doyle determined that the most likely 
place for measurable quantitative changes to emerge were in terms 
of how the teachers and students viewed their classrooms and 
themselves. Hence, non-cognitive "affective" variables were used 
exclusively to evaluate the degree to which ITPP affected the 
children in classrooms of Ford-trained teachers and to view the 
differences in Ford versus non-ForJ personnel. 



A iTicOor cor.C:ii.i fo^^ lLIj V^p, r is .in eval'-*ation of Doyle 
prijiaiv question: *'What difference does the Ford approach to training 
and placing professionals make in the lives of the ultirrate client — 
the youngster in the classroom?*' (p. 195) . Ihis is clearly an im- 
portant question, for even if all of the goals of trajning ap.d placer.ent 
were met satisfactorily but no differences in students wei*^ forthccTi- 
ing, the program would seem at least a partial failure. The question 
is, frcm an evaluational viewpoint, what changes might be expected, mea- 
surable, and most significant in terms of the program and the child? 
The nature of the effects FTPP should have had on inner-city children 
are implied in the 1967 article containing practical proposals. In 
particular, seme effect on the achie^/envent of the childr^en might not 
only be expected but desirable. Ifo^.^ever, Dcyle has argued that the cog- 
nitive grcwth of children touched by FTPP may not be apparent over the 
short' term^. A delayed effect extending over several years might show 
positive gains. Besides, cognitive gains are not clearly an explicit 
goal of the Ford ■ Program in operation . 

Design 

Overall, three separate sets . of data v;ere obtainod. 1) Mea- 
sures were made of a number of Ford and hon-Ford teachers ' attitudes and 
beliefs about the nature^^of^jtea^>ir^ 2) Students were asked to res- 
pond to instruments ^:ttempting to tap their academic self -concept -and 
their perceptions of the classrooms of vjhich they were a pai'^t. 3) fi- 
nally, a corpus of observational data was collected from Ford and non- 
Ford classrooi^. Ihe overall question confronted by these data was 
whether or not any systematic differences existed in the way FTPP tea- 
chers and students perceived themselves and fc-ehaved as compared with a 
sarr.ple of non-FTPP teachers. A seccr^d question v;as concemeu with 
d-ianges related to FTPP^s participation d'jring the 1971-72 acaderric 
year* Hence, these measures were made twice, early and late Ln that 
school year. 

As the Ford teachers had keen selected partly on the basis of 
certain personality characteristics considei-^ suitable for adequate 
Q participation in group work (as required by the r^ature of JTPP) , thiree 
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psychoToj^5.::::.n-1 :n?:rif?«jre^; v;ere ci^sc aririin-'.stered . These were used as in- 
dependent variables ir\ the analyses of sorre of the data. 

In s^jjnrriary , then, the design called far measures of Ford and 
non-Ford teachers and students on a number of paper-and -pencil , 
"effective" measures, as well as observational data from their class- 
rooms. These sets of data are descriJbed in detail below, but are 
listed here for convenient i>eferer.ce: 
Instrumentation 

I. Teacher Variables: 

A. InLepexxient Variables 
Dogmatism 

Flexibility/Rigidity 
Psychological Distance 

B. Dependent Variables 

Beliefs about Research • 
Receptivity to Curricular Change 

Teacher Conceptions of the Educational Pi^ocess 

Subject Matter Emphasis 
Personal Adjustnent 
Student Autononiy 
Emotional Disengagenient 
Consideration of Student Viewpoint 
Classroom Coxier 
Student Challenge 
Integrated Learning 

II. Student Variables; 

A, Dependent Variables 

AcadeTri.c Self -Concept 
Class Sentiment Index 

III. Observational Variables: 

Cognitive /MeiT>ory Critical Thir\^:ing Expi"^ssed Errotion Mgnt. 



Seeking 



Inforrning 



Accepti-ig 



Rejecting 
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The total popjlatior conL-if5ted of U8 s«sarKlary and .9 elenentary 
dassroons. The analyses have been procedins with the data frnn all 
schools , but the results given heru will focus on the King school . 

The design is that of u pr^- post -test of two sufc-groups frcn 

an essentially hcrrogejieoas oopulation. Thuo, the indicated analysis is 

tltrough t?ie use of analyc>is of covarf^ej CAncx)va), examining the post- 

g 

test scores ;ath the pre- tact covc.ri^ out. With respect to the King 
school data and for the purpcsec of this paper ^ the om^ hypothesis tested 
was that of treatment » in v/Iiich participation in FTPP is evaluated 
against a ccmparison group r^f nari-^Ford teachere and students. 

The argiOTent is: Jttvade that if ch3 Ford Program is to be consi- 
ders! "effective^', Ford tezchers, by tha end of the academic year, 
should differ frcm thos3 tQP.QtL^vj who hcv^ not b-^en exposed to the same 
training experience in thair beliefs about education. One set of ques- 
tions involved a cartpea>:\scn of Fcrc f^nd non-Ford teadiers as to certain 
peroepticns of cl«sroom act iv:* tier and thaSr r:£iactions to than. In 
particular > Ford traajied tecichsrs: 

. . .should perctiiva ^-^^atcr volue in classrocn 
resea.rch, l>e iroi-:; .-^ceptive to curriciilar change, 
not place mich infor tanco on rubject iratter, 
te iBDT*. ordentcd to^^D^d tha personal adjust- 
ment of cti.'d^-T>ts , t^tr^iss the ijnportance of the 
students* viewpoint, placo less importance on 
maintainiixg classrocrs raccording to es"^ ^blished 
rules and procedi.irjs , end pla::e greater cjmphasis 
on integrating content Vvdth tlie broader aspects 
of tlie studoiits* wrld^ so th-^t the children 

are aware, not only cf the facts of what they 

7 

leom, but also of tho Tn3aninc. (p* 216). 
In order to evaluate whether the above assertions were sup- 
portable, the adiftinistration cf a battery of different measures to tea- 
chers was no.cessary. Tr^ese arc briefly described here. First, a five- 
item scale, tenned "Beliefs About R?search^\ v;as piven to all teachers 
in the san5>le.* It requires the respondent to indicate on a five-point 
scale rangij>g frcsn "strcnr^ly rj^rca" to ^'L^trcrgly disagree" his res- 
ponses to such item^ as "There iz little research done that is of 
value to classroom tea<ihers»'' 



A second scale orir^inated wind staadardized by Wehling and 
Charters temed 'TTeacl^ Conceptions of ti>e Educational Process" and 
Including eight factor^ scales. 

1) Subject Matter Emphasis, a dimension which -neasured the 
teacher's relative attributiori of ijnpoxiiance to irorc traditional sub- 
ject matteJTS. 

2) Personal Adjustment Ideology, measuring the degree to which 
the teacher iiKiicates a concern with studcmts' probleris. 

3) . linotional Disengagem^t , a measure of social distance be- 
tween teacher and student. 

U) Student aUtonoriTy vs Teacher Direction, a bipolar factor 
dealing vdth the natijre of the teacher's management of the classrxxfn. 

5) Consideration of the Student Viewpoint, a measure of em- 
pathy • 

6) Classroom Ord.er, mcasxjring the degree to which the teacher 
perceives learning as taking plscc best in an atirosphere of order and 
deoarum. 

7) Student Challenge , a factor measuring the degree to which 
learning should be induced in students . 

8) Integrative L?amirif:5 ^ neasvire of the belief by the tea- 
cher in the stijdents' ed-iievixig en^.otional understanding in learning. 

The third instrumant used was one drafted by Eridgas, called 
"Teacher Receptivity to Change. ""^^ He concluded that his seven iterr\s 
validly discriminated betvnen teachers v;illing to ur 'ertake an innova- 
tive practice as against those not so inclined. 

With respect to differences in stiidemts taught by Ford as 
opposed to nan-Ford teachers, Doyle argued that: 

Fran the students' joint of viw, the teaching 
practices should be different (i.e.^ in Ford- 
taught rooms versus non-Ford). The students in 
Ford classrooms should judge tlie work to be 
more exciting. Ford teachers should use praise 
Tnore often to reinforce students positively; 
they s^iould enco'arac;e students Dore often to 
exp'5ress feelings about the teaching. The tea- 
chers should enccirage stijdents to express ideas 
that c^re diffr^rent fixriii their ov.r.j they should 
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talk less tlwi .crtlier teachers , should encourage 
students to lead discussions , should assign in- 
dividual and group projects more often, use 
small groups for special projects, and, in exa- 
' mlnations, should not test facts and memories 
exclusively, Tlie students in Ford classrocms 
should consider that their teachers have less 
rigid expectations for thejn ard are mare "crea- 
tive and open". (216-217) 
Students were asked to respond to tvro paper-^and-pencil in- 
struments jneasuring tliree variables. Each test was contposed of items 
developed at the Instrumental Objectives Exchange in Los Angeles. 
Items fox"^ the first instrument, the Class Sentijnent Index (CSI^^ 
were selected to measure two variables; the students* perceptions of 
their teachers' teaching behavior; and the students' perceptions of 

the teachers^ feelings about thonselves, .We called the second ins- 

12 

ti^ment the Academe Self -Concept Inventory (ASCI). Iteras for 
the ASCI vjere selected so as to measure the stjdents' self -concept 
in ths context of his class. 

For each instrument, exteasive item analysis was per- 
formed utilizing Lasarsfeld's Latant Structure or Latent Trait Analy- 
lU 

sis. In this particular application of Lazarsfeld's theory, the 
fonn of the function relating the underlying trait to the probability 
of a positive response on any item v;as assxiined to be the Normal Ogive. 
In Latent Trait Analysis the paraineters of this function are estimated 
and a test of fit produced for each item as well ^ as for the instr»i- 
ment as a wiiole. When an itm doesn't fit the model it is assumed 
that the responses to t!\at item were motivated by more than the one 
variable intended. In this way it^sns which do not appear to rw^asurc 
the sanve trait as the set of items as a whole can be eliminated and 
one is therefore able to obtain an instrument vhich is ineasuring 
essentially ono underlying trait. results of this analysis arc 
presented belcv?. 

A third set of data was obtained from j^n-participant ob- 
servers using an objective observational instrument called CERLI 
Verbal-Behavior Classification System (CVC).'''^ Doyle predicted that 



the observers "should rave noted differing classrxxiTi practices^' in Ford 

teachers' roc^rs ar^ carui'^ared isrith ncn-^brd roams. Explicitly, ncn- 

paorticipant obseiA^ers should have noted "differir.^ classrocn practices/' 

as follows: ^*ford teacher? tcil;<: less than other tc chex^, use praise 

to reinforce students positively, seek fewer responses thiat call for 

16 

critical thinking (217) Finally, stpjdents in Ford -taught classes 
might well give less factual infonnation, more critical thought, ex- 
press JBore positive sentijrient toward their classes and higher esti- 
mates of their own academic perf orrrance . 

The CVC alloi^rs an observer to tally the percentage of tijT.e 
teadier and studej'it talk. In addition, all "talk" can be further bro- 
ken down to fit into one av a combination of 16 cells so that its 
qualities can be examned rriore precisely* In particular, all vertel 
activity in the classix>an can be coded acccirding to four content cate- 
gories — CogTiition TT^iory; productive, critical thinking; expressed 
emotion; management — and four categories of process used — seeking, 
informing, accepting, rejecting* l"hat is, each verbal exchange can be 
classified most grossly as to whether it was emitted by teacher or 
pupil, then rnore finely as to which of IB possible ccmbinaticns of pro- 
cess and content if belongs to. TTiUs it beccmes possible to test for 
differences in Ford and non-Ford classes as to which ones contain Trore 
teacher talk, seeking behavior, praisling of students, more critical 
thought, and so forth. 

For the purposes of this report, only six classrooms at Mar- 
tin L. King High School were available for analysis, Ford tea- 
dhet"^ and* four non-Ford teac>)ers were observed on several occasions 
Loth (sarly and late in the school year. Conparisons on all categori- 
z5±lohs of verbal behavior of interest 'to this study revealed no sig-- 
nif icant differences either in time when i^ieasUres were made or iii 
Ford vs non--Ford ■ ccrrtparisons . 
independent Variribles ' 

Effective participatioii in the Ford Prbgran requires, ac;cnR 
other things, a personality suitable for dose work in groups, and one 
wShich is open to iruiovative change in one's teaching, ' Acccrdir.jly 5 
selection Ford staff hopefully would be e>n>ected 

to produce a' group of teachers with "personality characteristics dif- ' 
ferent fron the general pof^ulation of teachers" • In order to test for* 
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the effectiveness of the selection process (and to get a handle on 
possible sources of bias in the study) all Ford teachers and a selec- 
tion of non-Ford teachiers at each school we:^ measured on three 
personality traits- It was assumed that their abilit:/ to work in 

groups might be iixJicated by their sonres on the Psychological Dis- 
17 

tance Scale, and thexr openness vo change by their scores on 



the Rokeach Dogmatism Scale E 
Scale. 
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and the Flexibility/Rigidity 



Table I 
Test of Group Differences 
CFord vs. Non-Ford) on the Paper-and-Pencil Measures 
Paper and Pencil Measurer 

Independent variables 



Univariate P 
Dogmatism 

Flexibility/Rigidity 
Psychological Distance 



NS 
- NS 
.01 

Teachers 



Dependent variable 



Receptivity to Curriculum Chaxige 
Attitude Towards Research 
Subject Matter Emphasis 
Personal Adjustment 
Stux3ent Ai itcncriy 
Ebotional DLsen^agement 
Consideration of Student: Viewpoint 
'Classroom- Order 
Student Challerige 
Integrated Learning 

St'jdents 

Dependent Variable 



CSX 



1. Teaching Behavior 

2. Feelings About Students 



NS 
.05 
NS 
NS 
NS 
NS 
NS 
NS 
NS 
NS 



NS 
NS 



Miltivariate P 



• 05 



NS 



J 



MS 



J 



NS 
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Results ajKl Discussion 

As we consider Table 1^ we note first that the teachers selec- 
ted for the Ford Program did in fact differ significantly on the three 
personality measures taken together, f ran those non-Ford teachers on 
which v.^ obtained data. However, before considering this result farther , 
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we wish to point out that there is a problem in its generalization. 
Biased selection of participants whi-:h preclude'^ the l>asic assump- 
tion of randomization underlying the p^eneral linear hypothesis (e.g, 
i^iCOVA) severely lijni.ts the general izaLilit7 of r>esultr . Sr.rply 
selecting teachers in a iranner such as that used in the Ford Train- 
ing and PlaceTient Program — one desipied to produce a group of tea- 
chers open to irjuovative change — and then giving this group a cer- 
tain sum of money to produce change might easily explain any dif fe- 
lloes in i'he students. Of cxwrse, we could (and, in fact, did, 
with no change in the results) covary out these independent variable 
in an attempt to control for tliis selection bias. But we are still 
in a situation of limited generalizafcility. Furthermore , other fac- 
tors relevant to the meas^ired outcomes, but unknown to the research- 
er, may be present, particularly in the affective dorain. 



Ford and non-Ford ter:.2hers on personality measures would seem to 
support the content:' on tfiat the selection process used in the Ford 
Training and Placement Program produced a group of teachers with 
characteristics different from the general population of teachers. 
However, the non-Ford t*.?.chers were self -selected in that th-e.-y 
a{9c>eed to be so meaeun^ .1. This is again a biased sample frx^m the 
relevr-it population (trls tiirte of non-Ford teachers) and in terms 
of confirmatory data analysis is again equi^cal. However, we would 
arg^je that removing this source of bias (i.e., including teachers 
vtfio v^re approached, but refused to participate) wx:>uld only effect 
the result obtained on the Itegmatism Scale, the Flexibility /Pi gidity 
Scale, or the Psychological Distance by incr^sing the observed dif- 
ference. Therefore, we conclude that the data collected tend to 
support the hypothesis that'^the^eleetiiDn process produces teachers 
with characteristics different from the generalj)opulation of 
.teachers. The result on the J^sycholceical Distance^ Scale would 
support particularly the contention that the selection process pro- 



duces a group of Teachers who possess at least crie^' characteristic 
deemed desirable for group work. ^"^^^ 



Of course, one can disagree with this argurnent :;fend vie. 
welccme such comnients); however, this possibility of disagreement 



The significant difference reported in Table I bet;-:een 
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allows us tc state our primary point with respect to this issue. The 
only way out of such an equivocal situation is, to randauize • In this 
context we sirnply note that there was one dependent variable vMch 
showed the expected result: Ford teachers* responses irdiccited a rrcre 
positive attitude toward research than non-Ford teachers' responses. 
As the training sessions and many other activities of the cadres were 
closely related to the strongly research oriented University of Cld- 
cago, such a result is not surprising. 

With respect to the students , we are in a much better pcsi- 
tion» at least .in terms of randcmization. While we might not be aile 
to attribute 'any t>rDrthwhile results to the Ford Program as such, the 
treatment was applied to a randan selection of students , and the treat- 
ment » selection process, or whatever, is essentially replicable. How- 
ever, the question of general izability becones moot. The student res- 
ponses to the subscales of the CSI concerning the students' percep- 
tions of tw aspects of their teachers' behavior show no significant 
differences between stude.^ts in the Ford and non-Ford classec. The 
Student Self -Concept Index, while yielding a very good fit in the 
elejnentary school population, proved to be inappropriate for the Mgh 
school stiidents. ^ 

In evaluating the data as collected (as opposed to the 
"quantitative nethod"), an additional conment seems in order, r'ap'^r-' 
and-pencil tests of affective traits are notorious for large error \^ari- 
ances. For example, ij^i all three student variables the range of 
scores was slightly les's than twice ti\e standard error. Therefore, 
v.tdle ut2 clearly are not in a position to reject the null hypothesis of 
no difference between groups, the inprc*cise nati'^r^e of these rreasures 
?nc?tivates the statisticians' primary admonition: one does not accept 
the null hypothesis, one fails to reject the null hypothesis • 
Discussion 

It should be cle^ar by now that the fcv; differences in the 
Ford teachers and students; as conparcd with non-Ford may mean manv 
tI'Ling.S! inappropriate r.easures, uTire liability of those used, toe 
small a saTiple'and so forth. One thing seems clear, however. The 
analyses of the FTPP as reported in this paper are wrong to the de- 

'•4 

gree that they indicate no ^lianges tcikijig place . The prcgran v.'rouriht 
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many and profaind changes, as tV-c other modes of evaluation ii^dicated. 
Of this there can be no '^buLt. Frrhfips our* c:?iortco?air;?, is that we, 
as psychometricians , have le»3;"i v/aodecl to the f./aditional paper-and- 
pencil approacl-) to evaluatior.. It is '^.iTiple ojiough to 'distribute a 
group of already constructed tests ar;d hope for 3ome differences to 
show up. Then can clairrt that the progi"'am vras the cause. It is 
far more difficult to discover those elements in the program that 
should result in change in the pra-^icipcints ai-id. to devise ways to 
quantitatively measure these. And fvu^uher, to adhere to traditional 
canons of msasureinarvt and test theory in doing it. Life was siirply 
too short to do that in the cas3 of rrPP. As oixr assistant director, 
Jim McCampbell, is fond of saying, ^'T^ie trouble x/i'cla this program is 
that it's always: a ys^ar be^vlnd itself." Lnd^ed, when we should liave 
b^en devisijig £:tpprGpr:'€vte jT.:?asures for the ffjial evaluation, we were 
in the midst of ^le of the countless cris3S tr^at determined the 
oourse of events, ^v?. pui'pcsc^s ^v.ire — e^-^' are — still evolving. 'A 
quantitative r.pprradi t}*:a'': f^.^ids a r?.tac.o.a3. i^r.sis in tlic nature of 
the FTPP itself is cUi'i'icvIt iu 3urh a fluid erivirci jT^nt . Ar.d with- 
out a firm grdp c:: th^ lir.v.-lcr:'.-.no ond arr-s of the progra^n and, in 
particular, the i..rocer-:jo-j L/ i-^^^ncl; -^h^-iisc c^r^ to hn achiovad, quanti- 
tative evaluation of th3 sort outlined her-^ cecr.vjs inappropriate. 
Wiat is requir^ed is perr.irx^iy a s:d::tinr;, rasper r>\ve means of secur- 
ing numerical data tJ-^at^ at the Sc^ivj time, is staxJle enough to be 
useful in decisioii-jn.*i^:u^:;v Tvb hav^^ net yet found thi^ . 

It sef^ms clcc-r that i;r.e 3Vr?Jiv\tion n'xiededj but lacking, in 
the FTPP required the fcr:Tation of a closer connection betveen that 
which is measured r:r:i the p'A^^p:v.r/n irt-?nt. This should be implied 
by this summary. We ccin sury^est L--.:i>e cr^cnpler:. of tlie sorts of com- 
parisor.s that were nac v^At: msxtiZ quc\ntitative d^ta, but vrere cle-^irly 
suggested by the Getnels' article. '^^ The frequency of professional 
interactions ainong Ford teachers could easily have been greater th-iJi 
that among non-Ford teacher,-:; p^-p-r-and -pencil tests of the teachers* 
Ja^wledre of the differ- nt rol^is that exi^t in tho social systom of 
schools x^x>uld have piXihjoly shcv.r, sigrdficant differences; and related 
to this knowledge, moi^^ effective u?.r- .the role specialists would 
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probably have been made by Ford teachers. To the extent that such re la 
tionships were not examined, we suppose that the FTPP is improperly 
represented by the present daU. Only when program and evaluation are 
specifically and firmly linked together will an adequate sumifttivc 
evaluation be possible. 
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